<!DOCTYPE html><html lang="zh-CN" data-theme="light"><head><meta charset="UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Java操作HDFS示例 | 时光荏苒</title><meta name="keywords" content="Hadoop,java"><meta name="author" content="Lancer代良晨"><meta name="copyright" content="Lancer代良晨"><meta name="format-detection" content="telephone=no"><meta name="theme-color" content="#ffffff"><meta http-equiv="Cache-Control" content="no-transform"><meta http-equiv="Cache-Control" content="no-siteapp"><meta name="description" content="1. 环境准备 大数据集群一套，没有的可以自己本地搭建一套（参考地址：https:&#x2F;&#x2F;www.jianshu.com&#x2F;p&#x2F;2c2ae6490fa0） 本地安装JDK 本地安装IDEA或者Eclipse  2. 创建Maven项目在IDEA工具中创建一个maven项目，并在pom.xml中添加以下依赖： 123456789101112131415161718192021222324252627282">
<meta property="og:type" content="article">
<meta property="og:title" content="Java操作HDFS示例">
<meta property="og:url" content="https://lancerdai.gitee.io/2020/02/19/Java%E6%93%8D%E4%BD%9CHDFS%E7%A4%BA%E4%BE%8B/index.html">
<meta property="og:site_name" content="时光荏苒">
<meta property="og:description" content="1. 环境准备 大数据集群一套，没有的可以自己本地搭建一套（参考地址：https:&#x2F;&#x2F;www.jianshu.com&#x2F;p&#x2F;2c2ae6490fa0） 本地安装JDK 本地安装IDEA或者Eclipse  2. 创建Maven项目在IDEA工具中创建一个maven项目，并在pom.xml中添加以下依赖： 123456789101112131415161718192021222324252627282">
<meta property="og:locale" content="zh_CN">
<meta property="og:image" content="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/img/default.jpg">
<meta property="article:published_time" content="2020-02-19T11:40:59.000Z">
<meta property="article:modified_time" content="2020-10-03T14:01:26.651Z">
<meta property="article:author" content="Lancer代良晨">
<meta property="article:tag" content="Hadoop">
<meta property="article:tag" content="java">
<meta name="twitter:card" content="summary">
<meta name="twitter:image" content="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/img/default.jpg"><link rel="shortcut icon" href="/img/favicon.png"><link rel="canonical" href="https://lancerdai.gitee.io/2020/02/19/Java%E6%93%8D%E4%BD%9CHDFS%E7%A4%BA%E4%BE%8B/"><link rel="preconnect" href="//cdn.jsdelivr.net"/><link rel="preconnect" href="//busuanzi.ibruce.info"/><link rel="stylesheet" href="/css/index.css"><link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free/css/all.min.css"><link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@fancyapps/fancybox@latest/dist/jquery.fancybox.min.css"><script>var GLOBAL_CONFIG = { 
  root: '/',
  hexoversion: '5.1.1',
  algolia: undefined,
  localSearch: undefined,
  translate: undefined,
  noticeOutdate: undefined,
  highlight: {"plugin":"highlighjs","highlightCopy":true,"highlightLang":true},
  copy: {
    success: '复制成功',
    error: '复制错误',
    noSupport: '浏览器不支持'
  },
  relativeDate: {
    homepage: false,
    post: false
  },
  runtime: '',
  date_suffix: {
    just: '刚刚',
    min: '分钟前',
    hour: '小时前',
    day: '天前',
    month: '个月前'
  },
  copyright: undefined,
  ClickShowText: undefined,
  lightbox: 'fancybox',
  Snackbar: undefined,
  justifiedGallery: {
    js: 'https://cdn.jsdelivr.net/npm/justifiedGallery/dist/js/jquery.justifiedGallery.min.js',
    css: 'https://cdn.jsdelivr.net/npm/justifiedGallery/dist/css/justifiedGallery.min.css'
  },
  isPhotoFigcaption: false,
  islazyload: false,
  isanchor: false
};

var saveToLocal = {
  set: function setWithExpiry(key, value, ttl) {
    const now = new Date()
    const expiryDay = ttl * 86400000
    const item = {
      value: value,
      expiry: now.getTime() + expiryDay,
    }
    localStorage.setItem(key, JSON.stringify(item))
  },

  get: function getWithExpiry(key) {
    const itemStr = localStorage.getItem(key)

    if (!itemStr) {
      return undefined
    }
    const item = JSON.parse(itemStr)
    const now = new Date()

    if (now.getTime() > item.expiry) {
      localStorage.removeItem(key)
      return undefined
    }
    return item.value
  }
}</script><script id="config_change">var GLOBAL_CONFIG_SITE = { 
  isPost: true,
  isHome: false,
  isHighlightShrink: false,
  isSidebar: true,
  postUpdate: '2020-10-03 22:01:26'
}</script><noscript><style type="text/css">
  #nav {
    opacity: 1
  }
  .justified-gallery img {
    opacity: 1
  }

  #recent-posts time,
  #post-meta time {
    display: inline !important
  }
</style></noscript><script>(function () {
  window.activateDarkMode = function () {
    document.documentElement.setAttribute('data-theme', 'dark')
    if (document.querySelector('meta[name="theme-color"]') !== null) {
      document.querySelector('meta[name="theme-color"]').setAttribute('content', '#0d0d0d')
    }
  }
  window.activateLightMode = function () {
    document.documentElement.setAttribute('data-theme', 'light')
    if (document.querySelector('meta[name="theme-color"]') !== null) {
      document.querySelector('meta[name="theme-color"]').setAttribute('content', '#ffffff')
    }
  }

  const autoChangeMode = 'false'
  const t = saveToLocal.get('theme')
  if (autoChangeMode === '1') {
    const isDarkMode = window.matchMedia('(prefers-color-scheme: dark)').matches
    const isLightMode = window.matchMedia('(prefers-color-scheme: light)').matches
    const isNotSpecified = window.matchMedia('(prefers-color-scheme: no-preference)').matches
    const hasNoSupport = !isDarkMode && !isLightMode && !isNotSpecified

    if (t === undefined) {
      if (isLightMode) activateLightMode()
      else if (isDarkMode) activateDarkMode()
      else if (isNotSpecified || hasNoSupport) {
        const now = new Date()
        const hour = now.getHours()
        const isNight = hour <= 6 || hour >= 18
        isNight ? activateDarkMode() : activateLightMode()
      }
      window.matchMedia('(prefers-color-scheme: dark)').addListener(function (e) {
        if (saveToLocal.get('theme') === undefined) {
          e.matches ? activateDarkMode() : activateLightMode()
        }
      })
    } else if (t === 'light') activateLightMode()
    else activateDarkMode()
  } else if (autoChangeMode === '2') {
    const now = new Date()
    const hour = now.getHours()
    const isNight = hour <= 6 || hour >= 18
    if (t === undefined) isNight ? activateDarkMode() : activateLightMode()
    else if (t === 'light') activateLightMode()
    else activateDarkMode()
  } else {
    if (t === 'dark') activateDarkMode()
    else if (t === 'light') activateLightMode()
  }
})()</script><meta name="generator" content="Hexo 5.1.1"></head><body><div id="mobile-sidebar"><div id="menu_mask"></div><div id="mobile-sidebar-menus"><div class="mobile_author_icon"><img class="avatar-img" src="/null" onerror="onerror=null;src='/img/friend_404.gif'" alt="avatar"/></div><div class="mobile_post_data"><div class="mobile_data_item is-center"><div class="mobile_data_link"><a href="/archives/"><div class="headline">文章</div><div class="length_num">13</div></a></div></div><div class="mobile_data_item is-center">      <div class="mobile_data_link"><a href="/tags/"><div class="headline">标签</div><div class="length_num">16</div></a></div></div><div class="mobile_data_item is-center">     <div class="mobile_data_link"><a href="/categories/"><div class="headline">分类</div><div class="length_num">8</div></a></div></div></div><hr/><div class="menus_items"><div class="menus_item"><a class="site-page" href="/"><i class="fa-fw fas fa-home"></i><span> 主页</span></a></div><div class="menus_item"><a class="site-page" href="/archives/"><i class="fa-fw fas fa-archive"></i><span> 归档</span></a></div><div class="menus_item"><a class="site-page" href="/tags/"><i class="fa-fw fas fa-tags"></i><span> 标签</span></a></div><div class="menus_item"><a class="site-page" href="/categories/"><i class="fa-fw fas fa-folder-open"></i><span> 分类</span></a></div><div class="menus_item"><a class="site-page" href="javascript:void(0);"><i class="fa-fw fas fa-list"></i><span> List</span><i class="fas fa-chevron-down expand"></i></a><ul class="menus_item_child"><li><a class="site-page" href="/music/"><i class="fa-fw fas fa-music"></i><span> Music</span></a></li><li><a class="site-page" href="/movies/"><i class="fa-fw fas fa-video"></i><span> Movie</span></a></li><li><a class="site-page" href="/games/"><i class="fa-fw fas-game"></i><span> game</span></a></li><li><a class="site-page" href="/books/"><i class="fa-fw fas-books"></i><span> books</span></a></li><li><a class="site-page" href="/Gallery/"><i class="fa-fw fas-Gallery"></i><span> Gallery</span></a></li></ul></div><div class="menus_item"><a class="site-page" href="/link/"><i class="fa-fw fas fa-link"></i><span> 友链</span></a></div><div class="menus_item"><a class="site-page" href="/about/"><i class="fa-fw fas fa-heart"></i><span> 关于我</span></a></div></div></div></div><div id="body-wrap"><div id="sidebar"><i class="fas fa-arrow-right on" id="toggle-sidebar"></i><div class="sidebar-toc"><div class="sidebar-toc__title">目录</div><div class="sidebar-toc__progress"><span class="progress-notice">你已经读了</span><span class="progress-num">0</span><span class="progress-percentage">%</span><div class="sidebar-toc__progress-bar">     </div></div><div class="sidebar-toc__content"><ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#1-%E7%8E%AF%E5%A2%83%E5%87%86%E5%A4%87"><span class="toc-number">1.</span> <span class="toc-text">1. 环境准备</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#2-%E5%88%9B%E5%BB%BAMaven%E9%A1%B9%E7%9B%AE"><span class="toc-number">2.</span> <span class="toc-text">2. 创建Maven项目</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#3-%E7%BC%96%E5%86%99%E4%BB%A3%E7%A0%81"><span class="toc-number">3.</span> <span class="toc-text">3. 编写代码</span></a></li></ol></div></div></div><header class="post-bg" id="page-header" style="background-image: url(https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/img/default.jpg)"><nav id="nav"><span id="blog_name"><a id="site-name" href="/">时光荏苒</a></span><span id="menus"><div class="menus_items"><div class="menus_item"><a class="site-page" href="/"><i class="fa-fw fas fa-home"></i><span> 主页</span></a></div><div class="menus_item"><a class="site-page" href="/archives/"><i class="fa-fw fas fa-archive"></i><span> 归档</span></a></div><div class="menus_item"><a class="site-page" href="/tags/"><i class="fa-fw fas fa-tags"></i><span> 标签</span></a></div><div class="menus_item"><a class="site-page" href="/categories/"><i class="fa-fw fas fa-folder-open"></i><span> 分类</span></a></div><div class="menus_item"><a class="site-page" href="javascript:void(0);"><i class="fa-fw fas fa-list"></i><span> List</span><i class="fas fa-chevron-down expand"></i></a><ul class="menus_item_child"><li><a class="site-page" href="/music/"><i class="fa-fw fas fa-music"></i><span> Music</span></a></li><li><a class="site-page" href="/movies/"><i class="fa-fw fas fa-video"></i><span> Movie</span></a></li><li><a class="site-page" href="/games/"><i class="fa-fw fas-game"></i><span> game</span></a></li><li><a class="site-page" href="/books/"><i class="fa-fw fas-books"></i><span> books</span></a></li><li><a class="site-page" href="/Gallery/"><i class="fa-fw fas-Gallery"></i><span> Gallery</span></a></li></ul></div><div class="menus_item"><a class="site-page" href="/link/"><i class="fa-fw fas fa-link"></i><span> 友链</span></a></div><div class="menus_item"><a class="site-page" href="/about/"><i class="fa-fw fas fa-heart"></i><span> 关于我</span></a></div></div><span class="close" id="toggle-menu"><a class="site-page"><i class="fas fa-bars fa-fw"></i></a></span></span></nav><div id="post-info"><div id="post-title"><div class="posttitle">Java操作HDFS示例</div></div><div id="post-meta"><div class="meta-firstline"><span class="post-meta-date"><i class="far fa-calendar-alt fa-fw post-meta-icon"></i><span class="post-meta-label">发表于</span><time class="post-meta-date-created" datetime="2020-02-19T11:40:59.000Z" title="发表于 2020-02-19 19:40:59">2020-02-19</time><span class="post-meta-separator">|</span><i class="fas fa-history fa-fw post-meta-icon"></i><span class="post-meta-label">更新于</span><time class="post-meta-date-updated" datetime="2020-10-03T14:01:26.651Z" title="更新于 2020-10-03 22:01:26">2020-10-03</time></span><span class="post-meta-categories"><span class="post-meta-separator">|</span><i class="fas fa-inbox fa-fw post-meta-icon"></i><a class="post-meta-categories" href="/categories/%E5%A4%A7%E6%95%B0%E6%8D%AE/">大数据</a></span></div><div class="meta-secondline"> <span class="post-meta-separator">|</span><span class="post-meta-pv-cv"><i class="far fa-eye fa-fw post-meta-icon"></i><span class="post-meta-label">阅读量:</span><span id="busuanzi_value_page_pv"></span></span></div></div></div></header><main class="layout_post" id="content-inner"><article id="post"><div class="post-content" id="article-container"><h1 id="1-环境准备"><a href="#1-环境准备" class="headerlink" title="1. 环境准备"></a>1. 环境准备</h1><ol>
<li>大数据集群一套，没有的可以自己本地搭建一套（参考地址：<a target="_blank" rel="noopener" href="https://www.jianshu.com/p/2c2ae6490fa0">https://www.jianshu.com/p/2c2ae6490fa0</a>）</li>
<li>本地安装JDK</li>
<li>本地安装IDEA或者Eclipse</li>
</ol>
<h1 id="2-创建Maven项目"><a href="#2-创建Maven项目" class="headerlink" title="2. 创建Maven项目"></a>2. 创建Maven项目</h1><p>在IDEA工具中创建一个maven项目，并在pom.xml中添加以下依赖：</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">dependencies</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;<span class="name">dependency</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">groupId</span>&gt;</span>junit<span class="tag">&lt;/<span class="name">groupId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">artifactId</span>&gt;</span>junit<span class="tag">&lt;/<span class="name">artifactId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">version</span>&gt;</span>RELEASE<span class="tag">&lt;/<span class="name">version</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;/<span class="name">dependency</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;<span class="name">dependency</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">groupId</span>&gt;</span>org.apache.logging.log4j<span class="tag">&lt;/<span class="name">groupId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">artifactId</span>&gt;</span>log4j-core<span class="tag">&lt;/<span class="name">artifactId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">version</span>&gt;</span>2.8.2<span class="tag">&lt;/<span class="name">version</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;/<span class="name">dependency</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;<span class="name">dependency</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">groupId</span>&gt;</span>org.apache.hadoop<span class="tag">&lt;/<span class="name">groupId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">artifactId</span>&gt;</span>hadoop-common<span class="tag">&lt;/<span class="name">artifactId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">version</span>&gt;</span>2.7.2<span class="tag">&lt;/<span class="name">version</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;/<span class="name">dependency</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;<span class="name">dependency</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">groupId</span>&gt;</span>org.apache.hadoop<span class="tag">&lt;/<span class="name">groupId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">artifactId</span>&gt;</span>hadoop-client<span class="tag">&lt;/<span class="name">artifactId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">version</span>&gt;</span>2.7.2<span class="tag">&lt;/<span class="name">version</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;/<span class="name">dependency</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;<span class="name">dependency</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">groupId</span>&gt;</span>org.apache.hadoop<span class="tag">&lt;/<span class="name">groupId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">artifactId</span>&gt;</span>hadoop-hdfs<span class="tag">&lt;/<span class="name">artifactId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">version</span>&gt;</span>2.7.2<span class="tag">&lt;/<span class="name">version</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;/<span class="name">dependency</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;<span class="name">dependency</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">groupId</span>&gt;</span>jdk.tools<span class="tag">&lt;/<span class="name">groupId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">artifactId</span>&gt;</span>jdk.tools<span class="tag">&lt;/<span class="name">artifactId</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">version</span>&gt;</span>1.8<span class="tag">&lt;/<span class="name">version</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">scope</span>&gt;</span>system<span class="tag">&lt;/<span class="name">scope</span>&gt;</span></span><br><span class="line">			<span class="tag">&lt;<span class="name">systemPath</span>&gt;</span>$&#123;JAVA_HOME&#125;/lib/tools.jar<span class="tag">&lt;/<span class="name">systemPath</span>&gt;</span></span><br><span class="line">		<span class="tag">&lt;/<span class="name">dependency</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">dependencies</span>&gt;</span></span><br></pre></td></tr></table></figure>

<h1 id="3-编写代码"><a href="#3-编写代码" class="headerlink" title="3. 编写代码"></a>3. 编写代码</h1><p>新建名为HDFSClient的HDFS客户端操作类，下面是一些基础的客户端操作样例。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br><span class="line">284</span><br><span class="line">285</span><br><span class="line">286</span><br><span class="line">287</span><br><span class="line">288</span><br><span class="line">289</span><br><span class="line">290</span><br><span class="line">291</span><br><span class="line">292</span><br><span class="line">293</span><br><span class="line">294</span><br><span class="line">295</span><br><span class="line">296</span><br><span class="line">297</span><br><span class="line">298</span><br><span class="line">299</span><br><span class="line">300</span><br><span class="line">301</span><br><span class="line">302</span><br><span class="line">303</span><br><span class="line">304</span><br><span class="line">305</span><br><span class="line">306</span><br><span class="line">307</span><br><span class="line">308</span><br><span class="line">309</span><br><span class="line">310</span><br><span class="line">311</span><br><span class="line">312</span><br><span class="line">313</span><br><span class="line">314</span><br><span class="line">315</span><br><span class="line">316</span><br><span class="line">317</span><br><span class="line">318</span><br><span class="line">319</span><br><span class="line">320</span><br><span class="line">321</span><br><span class="line">322</span><br><span class="line">323</span><br><span class="line">324</span><br><span class="line">325</span><br><span class="line">326</span><br><span class="line">327</span><br><span class="line">328</span><br><span class="line">329</span><br><span class="line">330</span><br><span class="line">331</span><br><span class="line">332</span><br><span class="line">333</span><br><span class="line">334</span><br><span class="line">335</span><br><span class="line">336</span><br><span class="line">337</span><br><span class="line">338</span><br><span class="line">339</span><br><span class="line">340</span><br><span class="line">341</span><br><span class="line">342</span><br><span class="line">343</span><br><span class="line">344</span><br><span class="line">345</span><br><span class="line">346</span><br><span class="line">347</span><br><span class="line">348</span><br><span class="line">349</span><br><span class="line">350</span><br><span class="line">351</span><br><span class="line">352</span><br><span class="line">353</span><br><span class="line">354</span><br><span class="line">355</span><br><span class="line">356</span><br><span class="line">357</span><br><span class="line">358</span><br><span class="line">359</span><br><span class="line">360</span><br><span class="line">361</span><br><span class="line">362</span><br><span class="line">363</span><br><span class="line">364</span><br><span class="line">365</span><br><span class="line">366</span><br><span class="line">367</span><br><span class="line">368</span><br><span class="line">369</span><br><span class="line">370</span><br><span class="line">371</span><br><span class="line">372</span><br><span class="line">373</span><br><span class="line">374</span><br><span class="line">375</span><br><span class="line">376</span><br><span class="line">377</span><br><span class="line">378</span><br><span class="line">379</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> com.lancer.hdfs;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.io.File;</span><br><span class="line"><span class="keyword">import</span> java.io.FileInputStream;</span><br><span class="line"><span class="keyword">import</span> java.io.FileOutputStream;</span><br><span class="line"><span class="keyword">import</span> java.io.IOException;</span><br><span class="line"><span class="keyword">import</span> java.net.URI;</span><br><span class="line"><span class="keyword">import</span> java.net.URISyntaxException;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.conf.Configuration;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.BlockLocation;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.FSDataInputStream;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.FSDataOutputStream;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.FileStatus;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.FileSystem;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.LocatedFileStatus;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.Path;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.RemoteIterator;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.io.IOUtils;</span><br><span class="line"><span class="keyword">import</span> org.junit.Test;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">HDFSClient</span> </span>&#123;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 创建目录</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> Exception</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">mkdir</span><span class="params">()</span> <span class="keyword">throws</span> Exception,IOException,URISyntaxException </span>&#123;</span><br><span class="line">		Configuration conf = <span class="keyword">new</span> Configuration();</span><br><span class="line">		conf.set(<span class="string">&quot;fs.defaultFS&quot;</span>, <span class="string">&quot;hdfs://hadoop102:9000&quot;</span>);</span><br><span class="line">		<span class="comment">// 获取hdfs客户端对象</span></span><br><span class="line">		<span class="comment">//FileSystem fs = FileSystem.get(conf); </span></span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>)  , conf,  <span class="string">&quot;root&quot;</span>);</span><br><span class="line">		</span><br><span class="line">		<span class="comment">// 在hdfs上创建路径</span></span><br><span class="line">		fs.mkdirs(<span class="keyword">new</span> Path(<span class="string">&quot;/client/test4&quot;</span>));</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 关闭资源</span></span><br><span class="line">		fs.close();</span><br><span class="line">		</span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line">	</span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 上传文件</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">testCopyFromLocalFile</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException </span>&#123;</span><br><span class="line">		<span class="comment">// 1. 获取fs对象</span></span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>)  , <span class="keyword">new</span> Configuration(),  <span class="string">&quot;root&quot;</span>);</span><br><span class="line">		<span class="comment">//2 执行上传API</span></span><br><span class="line">		fs.copyFromLocalFile(<span class="keyword">new</span> Path(<span class="string">&quot;D:/学习/Hadoop权威指南（第四版）.pdf&quot;</span>), <span class="keyword">new</span> Path(<span class="string">&quot;/client/test&quot;</span>));</span><br><span class="line">		<span class="comment">// 3 关闭资源</span></span><br><span class="line">		fs.close();</span><br><span class="line">	&#125;</span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 上传文件，并设置备份数量</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">testCopyFromLocalFile2</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException </span>&#123;</span><br><span class="line"></span><br><span class="line">			<span class="comment">// 1 获取文件系统</span></span><br><span class="line">			Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">			<span class="comment">//configuration.set(&quot;dfs.replication&quot;, &quot;2&quot;);</span></span><br><span class="line">			FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line"></span><br><span class="line">			<span class="comment">// 2 上传文件</span></span><br><span class="line">			fs.copyFromLocalFile(<span class="keyword">new</span> Path(<span class="string">&quot;D:/学习/Hadoop权威指南（第四版）.pdf&quot;</span>), <span class="keyword">new</span> Path(<span class="string">&quot;/client/test3&quot;</span>));</span><br><span class="line"></span><br><span class="line">			<span class="comment">// 3 关闭资源</span></span><br><span class="line">			fs.close();</span><br><span class="line"></span><br><span class="line">			System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 文件下载</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">testCopyToLocalFile</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line"></span><br><span class="line">			<span class="comment">// 1 获取文件系统</span></span><br><span class="line">			Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">			FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line">			</span><br><span class="line">			<span class="comment">// 2 执行下载操作</span></span><br><span class="line">			<span class="comment">// boolean delSrc 指是否将原文件删除</span></span><br><span class="line">			<span class="comment">// Path src 指要下载的文件路径</span></span><br><span class="line">			<span class="comment">// Path dst 指将文件下载到的路径</span></span><br><span class="line">			<span class="comment">// boolean useRawLocalFileSystem 是否开启文件校验</span></span><br><span class="line">			fs.copyToLocalFile(<span class="keyword">false</span>, <span class="keyword">new</span> Path(<span class="string">&quot;/client/test3/Hadoop权威指南（第四版）.pdf&quot;</span>), <span class="keyword">new</span> Path(<span class="string">&quot;d:/Hadoop权威指南（第四版）.pdf&quot;</span>), <span class="keyword">true</span>);</span><br><span class="line">			</span><br><span class="line">			<span class="comment">// 3 关闭资源</span></span><br><span class="line">			fs.close();</span><br><span class="line">			System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 文件夹删除</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">testDelete</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 1 获取文件系统</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 2 执行删除</span></span><br><span class="line">		fs.delete(<span class="keyword">new</span> Path(<span class="string">&quot;/client/&quot;</span>), <span class="keyword">true</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 3 关闭资源</span></span><br><span class="line">		fs.close();</span><br><span class="line">		</span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 文件名修改</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">testRename</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 1 获取文件系统</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>); </span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 2 修改文件名称</span></span><br><span class="line">		fs.rename(<span class="keyword">new</span> Path(<span class="string">&quot;/README.txt&quot;</span>), <span class="keyword">new</span> Path(<span class="string">&quot;/README222.txt&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 3 关闭资源</span></span><br><span class="line">		fs.close();</span><br><span class="line">		</span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line">	</span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">testListFiles</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 1获取文件系统</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>); </span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 2 获取文件详情</span></span><br><span class="line">		RemoteIterator&lt;LocatedFileStatus&gt; listFiles = fs.listFiles(<span class="keyword">new</span> Path(<span class="string">&quot;/&quot;</span>), <span class="keyword">true</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="keyword">while</span>(listFiles.hasNext())&#123;</span><br><span class="line">			LocatedFileStatus status = listFiles.next();</span><br><span class="line">				</span><br><span class="line">			<span class="comment">// 输出详情</span></span><br><span class="line">			<span class="comment">// 文件名称</span></span><br><span class="line">			System.out.println(<span class="string">&quot;文件名：&quot;</span> + status.getPath().getName());</span><br><span class="line">			<span class="comment">// 长度</span></span><br><span class="line">			System.out.println(<span class="string">&quot;长度：&quot;</span> + status.getLen());</span><br><span class="line">			<span class="comment">// 权限</span></span><br><span class="line">			System.out.println(<span class="string">&quot;权限：&quot;</span> + status.getPermission());</span><br><span class="line">			<span class="comment">// 分组</span></span><br><span class="line">			System.out.println(<span class="string">&quot;分组：&quot;</span> + status.getGroup());</span><br><span class="line">				</span><br><span class="line">			<span class="comment">// 获取存储的块信息</span></span><br><span class="line">			BlockLocation[] blockLocations = status.getBlockLocations();</span><br><span class="line">			System.out.println(<span class="string">&quot;存储的块信息：&quot;</span> + status.getGroup());</span><br><span class="line">			<span class="keyword">for</span> (BlockLocation blockLocation : blockLocations) &#123;</span><br><span class="line">					</span><br><span class="line">				<span class="comment">// 获取块存储的主机节点</span></span><br><span class="line">				String[] hosts = blockLocation.getHosts();</span><br><span class="line">					</span><br><span class="line">				<span class="keyword">for</span> (String host : hosts) &#123;</span><br><span class="line">					System.out.println(host);</span><br><span class="line">				&#125;</span><br><span class="line">			&#125;</span><br><span class="line">				</span><br><span class="line">			System.out.println(<span class="string">&quot;-----------班长的分割线----------&quot;</span>);</span><br><span class="line">		&#125;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 3 关闭资源</span></span><br><span class="line">		fs.close();</span><br><span class="line">		</span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * HDFS文件和文件夹判断</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">testListStatus</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 1 获取文件配置信息</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> java.net.URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 2 判断是文件还是文件夹</span></span><br><span class="line">		FileStatus[] listStatus = fs.listStatus(<span class="keyword">new</span> Path(<span class="string">&quot;/&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="keyword">for</span> (FileStatus fileStatus : listStatus) &#123;</span><br><span class="line">			</span><br><span class="line">			<span class="comment">// 如果是文件</span></span><br><span class="line">			<span class="keyword">if</span> (fileStatus.isFile()) &#123;</span><br><span class="line">					System.out.println(<span class="string">&quot;f:&quot;</span>+fileStatus.getPath().getName());</span><br><span class="line">				&#125;<span class="keyword">else</span> &#123;</span><br><span class="line">					System.out.println(<span class="string">&quot;d:&quot;</span>+fileStatus.getPath().getName());</span><br><span class="line">				&#125;</span><br><span class="line">			&#125;</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 3 关闭资源</span></span><br><span class="line">		fs.close();</span><br><span class="line">		</span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * IO操作，上传文件，不使用封装好的方法</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">putFileToHDFS</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException </span>&#123;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 1 获取文件系统</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 2 创建输入流</span></span><br><span class="line">		FileInputStream fis = <span class="keyword">new</span> FileInputStream(<span class="keyword">new</span> File(<span class="string">&quot;D:/学习/Hadoop权威指南（第四版）.pdf&quot;</span>));</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 3 获取输出流</span></span><br><span class="line">		FSDataOutputStream fos = fs.create(<span class="keyword">new</span> Path(<span class="string">&quot;/Hadoop权威指南（第四版）.pdf&quot;</span>));</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 4 流对拷</span></span><br><span class="line">		IOUtils.copyBytes(fis, fos, configuration);</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 5 关闭资源</span></span><br><span class="line">		IOUtils.closeStream(fos);</span><br><span class="line">		IOUtils.closeStream(fis);</span><br><span class="line">	    fs.close();</span><br><span class="line">	    </span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * O操作，文件下载，不使用封装好的方法</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">getFileFromHDFS</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 1 获取文件系统</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 2 获取输入流</span></span><br><span class="line">		FSDataInputStream fis = fs.open(<span class="keyword">new</span> Path(<span class="string">&quot;/Hadoop权威指南（第四版）.pdf&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 3 获取输出流</span></span><br><span class="line">		FileOutputStream fos = <span class="keyword">new</span> FileOutputStream(<span class="keyword">new</span> File(<span class="string">&quot;d:/Hadoop权威指南（第四版）222.pdf&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 4 流的对拷</span></span><br><span class="line">		IOUtils.copyBytes(fis, fos, configuration);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 5 关闭资源</span></span><br><span class="line">		IOUtils.closeStream(fos);</span><br><span class="line">		IOUtils.closeStream(fis);</span><br><span class="line">		fs.close();</span><br><span class="line">		</span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 分块下载文件，第一块</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">readFileSeek1</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 1 获取文件系统</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 2 获取输入流</span></span><br><span class="line">		FSDataInputStream fis = fs.open(<span class="keyword">new</span> Path(<span class="string">&quot;/hadoop-2.7.2.tar.gz&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 3 创建输出流</span></span><br><span class="line">		FileOutputStream fos = <span class="keyword">new</span> FileOutputStream(<span class="keyword">new</span> File(<span class="string">&quot;d:/hadoop-2.7.2.tar.gz.part1&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 4 流的拷贝</span></span><br><span class="line">		<span class="keyword">byte</span>[] buf = <span class="keyword">new</span> <span class="keyword">byte</span>[<span class="number">1024</span>];</span><br><span class="line">			</span><br><span class="line">		<span class="keyword">for</span>(<span class="keyword">int</span> i =<span class="number">0</span> ; i &lt; <span class="number">1024</span> * <span class="number">128</span>; i++)&#123;</span><br><span class="line">			fis.read(buf);</span><br><span class="line">			fos.write(buf);</span><br><span class="line">		&#125;</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 5关闭资源</span></span><br><span class="line">		IOUtils.closeStream(fis);</span><br><span class="line">		IOUtils.closeStream(fos);</span><br><span class="line">	    fs.close();</span><br><span class="line">	    </span><br><span class="line">	    System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line">	</span><br><span class="line">	<span class="comment">/**</span></span><br><span class="line"><span class="comment">	 * 分块下载文件，第二块</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> IOException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> InterruptedException</span></span><br><span class="line"><span class="comment">	 * <span class="doctag">@throws</span> URISyntaxException</span></span><br><span class="line"><span class="comment">	 */</span></span><br><span class="line">	<span class="meta">@Test</span></span><br><span class="line">	<span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">readFileSeek2</span><span class="params">()</span> <span class="keyword">throws</span> IOException, InterruptedException, URISyntaxException</span>&#123;</span><br><span class="line"></span><br><span class="line">		<span class="comment">// 1 获取文件系统</span></span><br><span class="line">		Configuration configuration = <span class="keyword">new</span> Configuration();</span><br><span class="line">		FileSystem fs = FileSystem.get(<span class="keyword">new</span> URI(<span class="string">&quot;hdfs://hadoop102:9000&quot;</span>), configuration, <span class="string">&quot;root&quot;</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 2 打开输入流</span></span><br><span class="line">		FSDataInputStream fis = fs.open(<span class="keyword">new</span> Path(<span class="string">&quot;/hadoop-2.7.2.tar.gz&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 3 定位输入数据位置</span></span><br><span class="line">		fis.seek(<span class="number">1024</span>*<span class="number">1024</span>*<span class="number">128</span>);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 4 创建输出流</span></span><br><span class="line">		FileOutputStream fos = <span class="keyword">new</span> FileOutputStream(<span class="keyword">new</span> File(<span class="string">&quot;d:/hadoop-2.7.2.tar.gz.part2&quot;</span>));</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 5 流的对拷</span></span><br><span class="line">		IOUtils.copyBytes(fis, fos, configuration);</span><br><span class="line">			</span><br><span class="line">		<span class="comment">// 6 关闭资源</span></span><br><span class="line">		IOUtils.closeStream(fis);</span><br><span class="line">		IOUtils.closeStream(fos);</span><br><span class="line">		System.out.println(<span class="string">&quot;done&quot;</span>);</span><br><span class="line">		</span><br><span class="line">		<span class="comment">/**</span></span><br><span class="line"><span class="comment">		 * </span></span><br><span class="line"><span class="comment">		 * 在Window命令窗口中进入到目录E:\，然后执行如下命令，对数据进行合并</span></span><br><span class="line"><span class="comment">         * type hadoop-2.7.2.tar.gz.part2 &gt;&gt; hadoop-2.7.2.tar.gz.part1</span></span><br><span class="line"><span class="comment">         * 合并完成后，将hadoop-2.7.2.tar.gz.part1重新命名为hadoop-2.7.2.tar.gz。解压发现该tar包非常完整。</span></span><br><span class="line"><span class="comment">		 */</span></span><br><span class="line">	&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>
</div><div class="post-copyright"><div class="post-copyright__author"><span class="post-copyright-meta">文章作者: </span><span class="post-copyright-info"><a href="mailto:undefined">Lancer代良晨</a></span></div><div class="post-copyright__type"><span class="post-copyright-meta">文章链接: </span><span class="post-copyright-info"><a href="https://lancerdai.gitee.io/2020/02/19/Java%E6%93%8D%E4%BD%9CHDFS%E7%A4%BA%E4%BE%8B/">https://lancerdai.gitee.io/2020/02/19/Java%E6%93%8D%E4%BD%9CHDFS%E7%A4%BA%E4%BE%8B/</a></span></div><div class="post-copyright__notice"><span class="post-copyright-meta">版权声明: </span><span class="post-copyright-info">本博客所有文章除特别声明外，均采用 <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> 许可协议。转载请注明来自 <a href="https://lancerdai.gitee.io" target="_blank">时光荏苒</a>！</span></div></div><div class="tag_share"><div class="post-meta__tag-list"><a class="post-meta__tags" href="/tags/Hadoop/">Hadoop</a><a class="post-meta__tags" href="/tags/java/">java</a></div><div class="post_share"><div class="social-share" data-image="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/img/default.jpg" data-sites="facebook,twitter,wechat,weibo,qq"></div><link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/social-share.js/dist/css/share.min.css"><script src="https://cdn.jsdelivr.net/npm/social-share.js/dist/js/social-share.min.js" defer></script></div></div><nav class="pagination-post" id="pagination"><div class="prev-post pull-left"><a href="/2020/06/17/Volatile%E5%8F%AF%E8%A7%81%E6%80%A7%E4%BB%A3%E7%A0%81%E9%AA%8C%E8%AF%81/"><img class="prev-cover" src="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/img/default.jpg" onerror="onerror=null;src='/img/404.jpg'"><div class="pagination-info"><div class="label">上一篇</div><div class="prev_info">Volatile可见性代码验证</div></div></a></div><div class="next-post pull-right"><a href="/2020/02/17/Hadoop%E7%9A%84%E9%85%8D%E7%BD%AE%E6%96%87%E4%BB%B6%E4%B8%8E%E4%B8%89%E7%A7%8D%E8%BF%90%E8%A1%8C%E6%A8%A1%E5%BC%8F/"><img class="next-cover" src="http://p1.qhimg.com/bdr/__100/t018eb997ba237f590f.jpg" onerror="onerror=null;src='/img/404.jpg'"><div class="pagination-info"><div class="label">下一篇</div><div class="next_info">Hadoop的配置文件与三种运行模式</div></div></a></div></nav><div class="relatedPosts"><div class="headline"><i class="fas fa-thumbs-up fa-fw"></i><span> 相关推荐</span></div><div class="relatedPosts-list"><div><a href="/2020/02/16/Hadoop伪分布式环境配置与启动/" title="Hadoop伪分布式环境配置与启动"><img class="cover" src="http://p3.qhimg.com/bdr/__100/t012468d019e3ce466b.jpg"><div class="content is-center"><div class="date"><i class="far fa-calendar-alt fa-fw"></i> 2020-02-16</div><div class="title">Hadoop伪分布式环境配置与启动</div></div></a></div><div><a href="/2020/02/16/Hadoop的单机模式和官方示例/" title="Hadoop的单机模式和官方示例"><img class="cover" src="http://p5.qhimg.com/bdr/__100/t016591fff7fda9d11f.jpg"><div class="content is-center"><div class="date"><i class="far fa-calendar-alt fa-fw"></i> 2020-02-16</div><div class="title">Hadoop的单机模式和官方示例</div></div></a></div><div><a href="/2020/02/17/Hadoop的配置文件与三种运行模式/" title="Hadoop的配置文件与三种运行模式"><img class="cover" src="http://p1.qhimg.com/bdr/__100/t018eb997ba237f590f.jpg"><div class="content is-center"><div class="date"><i class="far fa-calendar-alt fa-fw"></i> 2020-02-17</div><div class="title">Hadoop的配置文件与三种运行模式</div></div></a></div><div><a href="/2020/02/15/Hadoop运行环境搭建教程/" title="Hadoop运行环境搭建教程"><img class="cover" src="http://p1.qhimg.com/bdm/1600_900_85/t013777ac0e30cc6227.jpg"><div class="content is-center"><div class="date"><i class="far fa-calendar-alt fa-fw"></i> 2020-02-15</div><div class="title">Hadoop运行环境搭建教程</div></div></a></div><div><a href="/2020/09/24/SpringMVC-WebSocket-实现群聊/" title="SpringMVC + WebSocket 实现群聊"><img class="cover" src="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/img/default.jpg"><div class="content is-center"><div class="date"><i class="far fa-calendar-alt fa-fw"></i> 2020-09-24</div><div class="title">SpringMVC + WebSocket 实现群聊</div></div></a></div><div><a href="/2020/06/17/Volatile可见性代码验证/" title="Volatile可见性代码验证"><img class="cover" src="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/img/default.jpg"><div class="content is-center"><div class="date"><i class="far fa-calendar-alt fa-fw"></i> 2020-06-17</div><div class="title">Volatile可见性代码验证</div></div></a></div></div></div></article></main><footer id="footer"><div id="footer-wrap"><div class="copyright">&copy;2020 By Lancer代良晨</div><div class="framework-info"><span>框架 </span><a target="_blank" rel="noopener" href="https://hexo.io">Hexo</a><span class="footer-separator">|</span><span>主题 </span><a target="_blank" rel="noopener" href="https://github.com/jerryc127/hexo-theme-butterfly">Butterfly</a></div></div></footer></div><section id="rightside"><div id="rightside-config-hide"><button id="readmode" type="button" title="阅读模式"><i class="fas fa-book-open"></i></button><button id="darkmode" type="button" title="浅色和深色模式转换"><i class="fas fa-adjust"></i></button></div><div id="rightside-config-show"><button id="rightside_config" type="button" title="设置"><i class="fas fa-cog"></i></button><button class="close" id="mobile-toc-button" type="button" title="目录"><i class="fas fa-list-ul"></i></button><button id="go-up" type="button" title="回到顶部"><i class="fas fa-arrow-up"></i></button></div></section><div><script src="https://cdn.jsdelivr.net/npm/jquery@latest/dist/jquery.min.js"></script><script src="/js/utils.js"></script><script src="/js/main.js"></script><script src="https://cdn.jsdelivr.net/npm/@fancyapps/fancybox@latest/dist/jquery.fancybox.min.js"></script><div class="js-pjax"><script async src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js"></script></div></div></body></html>